Skip to content

feat!: add generic e2e support to check-store workflow#279

Draft
digitalrisedorset wants to merge 12 commits into
graycoreio:mainfrom
digitalrisedorset:feat/check-store-e2e-support
Draft

feat!: add generic e2e support to check-store workflow#279
digitalrisedorset wants to merge 12 commits into
graycoreio:mainfrom
digitalrisedorset:feat/check-store-e2e-support

Conversation

@digitalrisedorset

Copy link
Copy Markdown

PR Checklist

Please check if your PR fulfills the following requirements:

PR Type

What kind of change does this PR introduce?

  • Bugfix
  • Feature
  • Code style update (formatting, local variables)
  • Refactoring (no functional changes, no api changes)
  • Build related changes
  • CI related changes
  • Documentation content changes
  • Other... Please describe:

What is the current behavior?

check-store.yaml currently supports:

  • coding-standard
  • unit-test
  • smoke-test

There is currently no generic mechanism for executing optional store-defined E2E test suites within the reusable workflow.

Fixes: N/A

What is the new behavior?

Adds optional e2e-test support to check-store.yaml.

The workflow now:

  • provisions a Node runtime
  • installs npm dependencies when a package.json exists
  • executes npm run test:e2e

This keeps E2E implementation details owned by the consuming store while allowing Mage-OS CI orchestration to execute generic Node-based E2E suites.

A minimal TypeScript-based E2E fixture was also added to validate the orchestration flow.

Does this PR introduce a breaking change?

  • Yes
  • No

Other information

This intentionally keeps the E2E contract lightweight and framework-agnostic for now.

The workflow orchestrates execution but does not enforce:

  • Playwright
  • Cypress
  • browser-specific tooling
  • E2E file structure conventions

This allows consuming stores to define their own E2E implementation details while reusing the existing Mage-OS orchestration and Magento service bootstrapping.

Comment thread .github/workflows/check-store.yaml
Comment thread .github/workflows/check-store.yaml Outdated
Comment thread .github/workflows/check-store.yaml
Comment thread .github/workflows/check-store.yaml Outdated
Comment thread .github/workflows/check-store.yaml Outdated
Comment thread .github/workflows/check-store.yaml
Comment thread _test/demo-package/Test/E2E/ItWorksTest.ts
Comment thread _test/demo-package/Test/E2E/tsconfig.json Outdated
Comment thread package.json Outdated
Comment thread _test/demo-package/Test/E2E/tsconfig.json Outdated
Comment thread .github/workflows/check-store.yaml Outdated
@damienwebdev

damienwebdev commented Jun 2, 2026

Copy link
Copy Markdown
Member

@herve

So, what I'd like to see is that we can run this against a store even if that store doesn't have the e2e tests installed.

So detect-e2e currently gates this against running if one isn't found, let's actually install the sample data and the e2e tests and attempt to run them.

@digitalrisedorset

digitalrisedorset commented Jun 2, 2026

Copy link
Copy Markdown
Author

@herve

So, what I'd like to see is that we can run this against a store even if that store doesn't have the e2e tests installed.

So detect-e2e currently gates this against running if one isn't found, let's actually install the sample data and the e2e tests and attempt to run them.

@damienwebdev I've pushed a follow-up commit.

The workflow now:

  • Continues normally when no E2E suite is present.
  • Installs sample data when E2E is enabled.
  • Installs the MageOS-adapted Playwright suite.
  • Executes the Playwright tests.
  • Removes the Node version pin.

I think this gets us much closer to the flow we discussed earlier today. Could you take another look when you have a moment? I suspect these changes might need tuning from where we are but this structure should be what we are aiming at. Once it works, I'll start 2 new activties:

  • adding more Playwrights test on a weekly basis to start adding genuine tests coverage (this is something anyone can add to as well)
  • start a E2E adapter so that any E2E tests suite can work. Lucas might want to join to add his test suite to work as an alternative

Comment thread .github/workflows/check-store.yaml Outdated
Comment thread .github/workflows/check-store.yaml Outdated
Comment thread .github/workflows/check-store.yaml Outdated
Comment thread .github/workflows/check-store.yaml Outdated
Comment thread .github/workflows/check-store.yaml
@digitalrisedorset

Copy link
Copy Markdown
Author

@damienwebdev I've removed the guards you mentioned. I understand this is so that we can discover fully the E2E execution path and identify what still needs to be wired up. I imagine we may reinstate some of these guards once E2E tests are fully integrated.

@damienwebdev

Copy link
Copy Markdown
Member

@Vinai asked two things:

  • Is it possible, to use the built-in playwright version vs. installing our own?
  • Is it possible to disable npm scripts to just lower the security surface concern?

@digitalrisedorset

Copy link
Copy Markdown
Author

@Vinai asked two things:

  • Is it possible, to use the built-in playwright version vs. installing our own?
  • Is it possible to disable npm scripts to just lower the security surface concern?

@Vinai , @damienwebdev My thoughts were that MageOS would own the E2E tests and the execution environment (Node, Playwright version, dependency installation, etc.) would be a separate concern.
I understand the concern around keeping the Node/npm security surface area as small as possible, and I'm happy to investigate those options once we've validated the E2E flow itself.

@damienwebdev damienwebdev force-pushed the feat/check-store-e2e-support branch from 0b25c7f to 4bdd58b Compare June 10, 2026 13:18
@damienwebdev

damienwebdev commented Jun 10, 2026

Copy link
Copy Markdown
Member

@digitalrisedorset

Thanks for the continued work on this. E2e test code is now running. Before I re-review, I need to see a green run of the Check Store Test workflow executed in your own fork, since fork PRs here don't run the full CI automatically.

Here's how to run it on digitalrisedorset/github-actions-magento2:

  1. Enable Actions on your fork, go to the Actions tab and click "I understand my workflows, go ahead and enable them" if you haven't already.
  2. Push your changes to feat/check-store-e2e-support on your fork.
  3. Trigger the workflow, either:
    • Actions tab → "Check Store Test" → "Run workflow" → select branch feat/check-store-e2e-support, or
    • open a draft PR within your fork (feat/check-store-e2e-supportdigitalrisedorset:main) so the pull_request trigger fires on every push.

Two notes:

  • The pin in check-store.yaml pointing resolve-check-config at digitalrisedorset/github-actions-magento2/resolve-check-config@feat/check-store-e2e-support is correct for validation, it's what makes the fork run exercise your modified action instead of the published one on main. Keep it while you iterate, but it must be reverted to graycoreio/github-actions-magento2/resolve-check-config@main before this can merge.

Once you have a green run, link it here and I'll pick the review back up.

@digitalrisedorset digitalrisedorset marked this pull request as draft June 10, 2026 16:01
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants